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© Audio/video synchronization in a digital transmission system. 



CM 
< 

CO 

o 

00 
CD 



© A receiver (FIGURE 2) for decoding associated 
compressed video and audio information compo- 
nents transmitted in mutually exclusive "frames" of 
data with respective time stamps PTS v id and PTS au( j 
respectively, includes a controller (216)' which is 
responsive to the respective received time stamps to 
provide coarse synchronization by delaying or skip- 
ping respective frames of one or the other of the 
components to approximately time align the two 
components. Fine synchronization is provided by 
adjusting the processing or clock frequency (2 15) of 
the audio signal processor (212) independent of the 
video processor(214). The control for the frequency 
adjustment is related to the difference between audio 
and video time stamps. 



vrsrtM 



MVEJU8 
TfUiePOffT 
P*OC£S£OA 



COWTHOL 
BUS 



CLOCK 



AUDtO 




DECO*JPft£SSOft 





|SJOP 



r 



VIOCO 

oecowpfttssoft 






^ . 




FIG. 2 



a. 

UJ 



Rank Xerox (UK) Business Services 

0. 10/3 09/3. J. " 



EP 0 648 056 A2 



This invention relates to a method and appara- 
tus for synchronizing the audio and video compo- 
nent signals at an audio/video signal decompres- 
sion apparatus 

Audio and video (A/V) transmission and recep- 
tion requires that the audio and video components 
be properly synchronized. EIA standards RS-250-B 
limit the time differential between associated audio 
and video signals to 25 ms. lead or 40 ms. lag 
time. Rim standards limit the time differential of 
associated audio and video to ±1/2 frame, which 
corresponds to 20.8 ms. An acceptable goal for 
source to viewer A/V synchronization is therefore a 
time differential of ±20 ms. 

Digital communication systems typically time- 
multiplex associated signal components over a sin- 
gle channel. Such multiplexing is common among 
A/V transmission systems proposed and imple- 
mented for cable, fiber, terrestrial and satellite ap- 
plications. The time multiplexing of the signal com- 
ponents may destroy their natural time relation- 
ships between the transmission and display of the 
information, Therefore, time critical components of 
the transmitted component signals may be asso- 
ciated with a time reference before being mul- 
tiplexed. This is referred to as "stamping" the 
information, and timing samples are referred to as 
time stamps. The receiver may then output the 
respective components in time relative to their re- 
spective time stamps. However to accomplish this 
the receiver must maintain a very precise local 
time reference, which is synchronous to the en- 
coder time reference. 

One reason that the receiver must be tightly 
coupled to the time base of the transmitter is to 
insure that the output of real time data matches 
that of the input to the receiver. If the receiver 
provides (displays) the data too rapidly, buffers in 
the receiver may underflow resulting in an interrup- 
tion of the output signal. If the receiver outputs the 
data too slowly . the buffers may overflow (assum- 
ing finite rate buffers) resulting is a loss of data. 

In one proposed system the receiver is syn- 
chronized to the transmitter by supplemental time 
stamps (system clock references, SCR) associated 
with predetermined packets of transmitted informa- 
tion. The timing of the capture of the time stamps. 
SCR, bear no relation to presentation time stamps 
(PTS) which are related to the video data, other 
than by virtue of being derived from the same 
counter. The SCR codes are generated by sam- 
pling a modulo 2 N counter (N * 32) which counts a 
substantially constant frequency crystal clock at the 
transmitter. The receiver incorporates a phase 
locked loop which has a free running frequency 
substantially equal to the frequency of the clock in 
the transmitter. The receiver clock (local clock) is 
also counted modulo 2 N , and each time a SCR 



arrives at the receiver the local counter is sampled 
to provide a local clock reference or LCR. No 
attempt is made to force the LCR to equal the 
SCR. Rather the local clock is adjusted based upon 
s processing changes in the difference between the 
LCR and SCR time stamps. An error signal is 
generated according to the relation 

ERR = |SCR„-SCR B .,| - | LCR n - LCR„.,| 

JO 

The signal ERR is utilized to control the local clock 
frequency. Via this process the LCR can be made 
arbitrarily close to the transmitter clock frequency. 
Note that since both the system and local clocks 
j 5 are counting modulo N. they periodically wrap ar- 
ound. On these occurrences the respective terms 
SCR„-SCR 0 . t and LCR„ -LCR„-,will be negative 
and erroneous. The system monitors the polarity of 
the respective differences and when one of the 
20 differences is negative the difference is ignored. 

Video signal coded according to the MPEG 
standard includes presentation time stamps, 
PTSytd, which are synchronized to the input video 
frames. The respective PTS vid indicate the relative 
25 times that the respective frames are . to be dis- 
played at the receiver, nominally 30 Hz, for NTSC 
source material. Associated audio is also encoded 
with presentation time stamps PTS aud based on the 
same time base as the system time and which time 
30 stamps are placed in an MPEG system packet 
layer encompassing the encoded audio data. An 
audio system packet layer may contain several 
"frames" of audio data and respective frames 
equal, in this example, 24 ms. of original audio 
55 data. Audio frames are approximately six times the 
duration of a (127 byte) transport packet. (Informa- 
tion to be transmitted, audio, video, data, etc. are 
segmented into respective transport packets of pre- 
determined size, with a variety of control words 
40 appended, to provide an extra layer of error correc- 
tion/detection and synchronization.) In addition, ac- 
cording to the MPEG protocol, the number of audio 
frames per MPEG system layer is a variable. 
Hence there may be little or no correlation between 
^5 the video PTS V i d and audio PTS aud presentation 
time stamps for associated audio and video source 
material. Thus synchronizing the audio and video 
components is difficult if one attempts to do so by 
comparing the PTSv* with the PTS, ud . It is an 
so object of the present invention to simplify the pro- 
cess of synchronizing associated audio and video 
components. 

A receiver for decoding associated video and 
audio information components transmitted in mutu- 
55 ally exclusive "frames" of data with respective time 
stamps PTSvid and PTS» od respectively, includes a 
controller which is responsive to the respective 
received time stamps to provide coarse synchro- 
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nization by delaying or skipping respective frames 
of one or the other of the components to approxi- 
mately time align the two components. Fine syn- 
chronization is provided by adjusting the process- 
ing or clock frequency of the audio signal proces- 
sor independent of the video processor. The con- 
trol for the frequency adjustment is related to the 
difference between audio and video time stamps. 

BRIEF DESCRIPTION OF THE DRAWINGS 



FIGURE 1 is a block diagram of audio/video 
compression apparatus. 

FIGURE 2 is a block diagram of a audio/video 
decompression apparatus embodying the present 
invention. 

FIGURE 3 s a block diagram of apparatus tor 
providing the receiver system clock signal having 
' substantially the same rate as the system clock of 
the compression apparatus. 

FIGURE 4 is a flow chart of the operation of 
the FIGURE 2 apparatus. 

FIGURES 5 and 6 are block diagrams of alter- 
native audio processing clock signal generation 
which may be implemented for element 215 in 
FIGURE 2. 

FIGURE 1 illustrates a typical system in which 
the invention may be practiced, which system is a 
compressed digital video signal transmission ar- 
rangement. In this system, video signal from a 
' f source 10 is applied to a video signal compression 
"- element 1 1 which may include a motion com- 
pensated predictive encoder utilizing discrete co- 
sine transforms. Compressed video signal from the 
element 11 is coupled to a formatter 12. The for- 
matter arranges the compressed video signal and 
other ancillary data according to some signal pro- 
tocol such as MPEG, a standard developed by the 
International Organization for Standardization (Or- 
ganisation Internationale De Normalisation). The 
standardized signal is applied to a transport pro- 
cessor 13. which divides the signal into packets of 
data and adds certain overhead to provide some 
noise immunity for the transmission purposes. The 
transport packets, which normally occur at a non- 
uniform rate are applied to a rate buffer 14 which 
provides output data at a relatively constant rate 
conducive to efficient use of a relatively narrow 
bandwidth transmission channel. The buffered data 
is coupled to a modem 1 5 which performs the 
signal transmission. 

A system clock 22 provides clocking signal to 
operate much of the apparatus, at least including 
the transport processor. This clock will operate at a 
fixed frequency such as 27 MHz for example. As 
shown herein, however, it is used to generate tim- 
ing information, The system clock is coupled to the 
. clock input of a counter 23 which may be arranged 



,o count modulo 2" . for example. The count 
values output by the counter are applied to two 
latches 24 and 25. The latch 24 is conditioned by 
the video source to latch count values on the 
s occurrence of respective frame intervals. These 
count values are denoted presentation t.me 
stamps. PTS's. and are included in the com- 
pressed video signal stream by the formatter 12. 
and are used by the receiver to provide lip-syn- 
,o chronization of associated audio and video informa- 
tion The latch 25 is conditioned by the transport 
processor 13 (or the system controller 21) to latch 
count values according to a predetermined sched- 
ule These count values are denoted system clock 
r5 references. SCR's. and are embedded as auxiliary 
data within respective auxiliary transport packets. 

Audio signal associated with the video signal 
from source 10 is applied to an audio signal com- 
pressor 18. The compressor 18 provides frame 
20 sampling pulses (independent of video frames) to 
control a latch 19. Responsive to the sampling 
pulses, the latch 19 captures count values provided 
by the counter 23. These latched values corre- 
spond to audio presentation time stamps PTS. ud . 
25 The PTS au «, are incorporated in the compressed 
audio signal provided by the compressor 18. The 
compressed audio signal is coupled to a transport 
processor 17 which divides the signal into packets 
of data and adds certain overhead to provide some 
30 noise, immunity for transmission purposes. The 
audio transport packets provided by the processor 
17 are coupled to a multiplexor 16 which time 
division multiplexes the audio and video transport 
packets. In the figure, separate transport proces- 
3S sors are shown in the audio and video signal pro- 
cessing channels. For systems wherein the data 
rate is moderate, the functions of the two transport 
processors and the multiplexor 16 may be sub- 
sumed in a single transport processor. 
<o The system controller 21 is a variable state 
machine programmed to coordinate the various 
processing elements. Note that the controller 21. 
the compressors 11 and 18. the transport proces- 
sors 13 and 17. and the rate buffer 14 may or may 
<s not operate synchronously via a common clock.ng 
arrangement as long as proper handshak.ng <s pro- 
vided between processing elements. However, he 
two compressors both derive PTS values from the 
same reference counter 23. thus a precise t.m.ng 
so relationship between the two compressed signals is 
provided in the compressed output signal. 

FIGURE 2 illustrates an exemplary receiver ap- 
paratus embodying the invention wherein the 
modem 200 performs the inverse 'action of the 
S5 modem 15 and the rate buffers 204 and 206 per- 
form in effect, the inverse function of the rate 
buffer 14. In FIGURE 2 a single inverse transport 
processor 202 is indicated, which processor div.des 



3 



s 



BP 0 648 056 A2 



6 



respective transport packets by service and al- 
locates the respective data to the appropriate pro- 
cessmg channel. In so doing the respective trans- 
port packet signal payloads are separated from the 
auxiliary data, with the respective payloads being 
applied to the appropriate processing channel and 
the auxiliary data applied to the system controller 
210. In an alternative arrangement a separate trans- 
port processor may be included in each processing 
channel and arranged to recognize and process 
only data associated with the respective channels. 

Compressed video data from the inverse trans- 
port processor 202 is applied to the rate buffer 204 
which provides compressed video signal according 
to the system protocol to the decompressor 214 
The rate buffer 204 accepts data at a bursty or 
inconsistent rate, and provides data on demand to 
the decompressor 214. The decompressor, respon- 
sive to the compressed video signal, generates 
non-compressed video signal for display or for 
storage etc.. in appropriate display or storage de- 
vices (not shown). 

Compressed audio data from the inverse trans- 
port processor 202 is applied to the rate buffer 206 
which provides compressed audio signal according 
to the system protocol to the audio decompressor 
212. The decompressor. 212. responsive to the 
compressed audio signal generates non-com- 
pressed audio signal for reproduction or for storage 
etc.. in appropriate speakers or storage devices 
(not shown). 

SCR^L^r 6 processor 202 also provides 

:Zlt I 3UXiIiary tranSport data ' and control 

signals to a system clock generator 208. The clock 

LTrV e ! POnSiVe t0 th6Se si 9 na,s Qenerates a 
system clock s.gnal synchronous with at least the 
transport processor operation. This system clock 
signal applied to the receiver system controller 
element? timi ' nQ ° f appropriate processing 

Figure 3' illustrates details of an exemplary 

mnn re ^ 6 ' at0f 208 * Data from th * receiver 
modem 200 is coupled to the inverse transport 
processor 202'. including an auxiliary packet detec- 
tor 31. The inverse transport processor 202' sepa- 

^LT'T h6ader d3ta ff0m the res P**ve 
^ T f BCket Pay,03ds - R «Ponsive to the trans- 
port header data, the processor 202' applies audio 
and video s.gnaf payloads to, for example, respec 
t ve decompression apparatus (not shown,, and 
aux^ ry d ata (designated as AUX) (0 > 

pnate auxihary data processing elements (not 
data are routed and stored in a memory elemen? 

a ' ia ,7 P3Cket det6Ctor 31 - whic * ™y be 

codl?i , " ter afranged t0 the 
codewords des.gnating an auxiliary transport pack! 



et containing a SCR. produces a control pulse on 
the occurrence of transport packets containing 
such data. The control pulse is utilized to capture 
and store within a latch 35, the count value cur- 
s rently exhibited by the local counter 36 at a time 
precisely related to the time of detection. The local 
counter 36 is arranged to count pulses provided by 
a voltage controlled oscillator 37. The counter 36 is 
arranged to count modulo M which may be. but is 
w not necessarily, the same number as its' coun- 
terpart counter in the encoder (counter 23). if m 
differs from N the difference may be accommo- 
dated in the error equation. 

The voltage controlled oscillator 37 i s con- 
is trolled by a low pass filtered 38 error signal pro- 
vided by a clock controller 39. The error signal is 
generated in the following fashion. Let the SCR 
arriving at time n be designated SCR n and the local 
count value concurrently captured in the latch 35 
20 be designated LCR„. The clock controller reads the 
successive values of SCR's and LCR's and forms 
an error signal E proportional to the differences 



25 



E => |SCR n - SCR,,., | - |LCR n - LCR„., 



The error signal E, is utilized to condition the 
voltage controlled oscillator 37 to a frequency ten- 
ding to equalize the differences. As indicated pre- 
viously, negative differences exhibited due to 
30 modulo counter wrap around; may be ignored. The 
error signal produced by the clock controller 39 
may be in the form of a pulse width modulated 
signal, which may be rendered into an analog error 
signal by implementing the low pass filter 38 in 
os analog components. 

The constraints on this system are that the 
counters at the two ends of the system count the 
same frequency or even multiples thereof. This 
requires that the nominal frequency of the voltage 
<o controlled osciilator be fairly close to the frequency 
of the system clock at the encoder. 

The foregoing approach provides rather rapid 
synchronization but may introduce a long term 
error. Long term error LTE is proportional to the 
<s difference 

LTE => |LCR n - LCRo| - |SCR n - SCRo| 

where SCRo and LCRo are for example the first 
so occurring SCR and the corresponding latched val- 
ue of the receiver counter. Nominally the error 
signals E and LTE will vary in discrete steps. As 
such, once the system is "synchronized'' the error 
signal will dither one unit about the null point. The 
55 preferred method of synchronization is to initiate 
control of the voltage controlled oscillator using the 
error signal E until a one unit dither occurs in the 
error signal E. and then to switch to the use of the 
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long term error signal LTE to control the voltage 
controlled oscillator. 

The system clock signal provided by the VCXO 
37 may be utilized to operate at least the transport 
processor and rate buffers. Since it is synchronized 
at least . in frequency with the encoder system 
clock, the possibility of rate buffer overflow or un- 
derflow due to clock timing errors is substantially 
non-existent. 

Refer again to FIGURE 2 for an explanation of 
audio/video synchronization. Recall that a presenta- 
tion time stamp PTS V id. is included in the com- 
pressed video signal associated with predeter- 
mined video data. The PTS vid is indicative of the 
relative time that the associated video is to be 
displayed. Similarly the compressed audio signal 
includes presentation time stamps PTS aud asso- 
ciated with audio to be reproduced at times related 
to the respective PTS aud . At the receiver the 
PTS, ud and PTS^ may not be compared directly 
to provide A/V synchronization because the respec- 
tive samples were determined at different instants. 
The respective PTS* values are compared to a 
continuous time base which is the receiver clock 
provided by the VCXO 37. This is done by sam- 
pling local time stamprs of the count values LCR. 

When data associated with an associated PTS 
is presented, The LCR is sampled. For example, 
The audio decompressor 212 issues a PTS aud 
' when a respective audio frame is output for re- 
production. At these times a control signal con- 
ditions the latch 220 to sample the LCR, the values 
of which will be designated LAS, for local audio 
stamp. Similarly when the video decompressor pro- 
vides a video frame for display, it provides a PTS vld 
and a control pulse to condition a latch 222 to store 
the current value of the LCR. These LCR values 
are designated LVS for local video stamps. 

The LAS and the corresponding PTS aud are 
coupled to respective input terminals of a sub- 
tracter 218 which develops the signal A a -pts ac- 
cording to the relation; 

A a- pts = PTS oud * LAS 

The LVS and the corresponding PTSvid are coup- 
led to respective input terminals of .the subtracter 
217 which develops the signal a v -pts according to 
the relation; 

Av- pts = PTS vid - LVS. 

The signals Av-pts and A a- pts are coupled to 
respective input terminals of a further subtractor, 
219, which develops an A/V synchronization error 
signal ERRpts according to the relation; 

ERRpts = Av-pts • A A- PTS 



Synchronization of the audio and video requires 
that the AA/ synchronization error be driven to 
zero. This indicates that when the difference in the 
5 values of the corresponding audio and video PTS's 
equals the time, in units of the local reference, 
between the occurrence of the corresponding 
PTS's. the audio and video signal will be in syn- 
chronization. 

io Two mechanisms may be used to adjust the 
A/V synchronization based upon the error signal 
ERRpts; ski PS and repeats of data sections and 
conversion clock deviation. Skipping fixed intervals 
or "frames" of audio advances the audio data 
;5 stream by a fixed interval relative to the video 
signal. Repeating (or muting without consuming 
data) delays the audio data stream by fixed inter- 
vals relative to the video signal. Skipping and re- 
peating audio frames is audible under many con- 
20 ditions. and therefore is only utilized for coarse 
adjustment of synchronization. Even so. brief skip- 
ping or repeating may be preferable to discernible 
audio/video synchronization errors. If the audio 
frames are less than 40 msec, coarse adjustment 
25 by skipping/repeating may result in synchronization 
errors within t20 msec, which is within industry 
standards for A/V synchronization. However . this 
synchronization will degrade if the audio conversion 
time base does not match that of the source. Once 
30 synchronization is coarsely adjusted, variations of 
the audio conversion clock frequency are arranged 
to further refine the A/V synchronization. 

The error signal ERRpts is applied to a filter 
and processing element 216. A filter function there- 
35 in smoothes the signal ERRpts to minimize aber- 
rant effects that might otherwise be generated by 
signal noise. The processing function in the ele- 
ment 216 then examines the smoothed error signal 
and determines whether a skip/repeat of audio 
40 should be utilized to effect coarse synchronization 
of the audio and video signals and/or whether an 
adjustment to the audio processing frequency 
should be utilized to effect fine synchronization. If a 
coarse synchronization adjustment is determined to 
4$ be necessary, the processor 216 provides a control 
signal to the audio decompressor 21 2 to condition 
the decompressor to skip or repeat the current 
decompressed audio frame. Alternatively, or in ad- 
dition to the coarse adjustment, if a fine adjustment 
so is determined to be necessary, the processor 216 
provides a control signal to the audio time base 
215 to adjust the frequency of the audio processing 
clock signal. 

The processing algorithm is detailed in the flow 
55 chart of FIGURE 4. After initialization 400 of the 
system, which is designated START, the system 
monitors 401 the audio decompressor for the oc- 
currence of a PTS, ud and if a PTS aud is detected it 
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is read 403 and a local clock reference LAS is 
captured and stored. If a PTS 8ud has not occurred, 
the system monitors the video compressor for a 
PTSv«, 402. If a PTS vid has occurred, the PTSvi d is 
read and a local clock reference LVS is captured 
and stored 404. When both a PTS aud and a PTS vid 
have been read the ERR PTS is calculated 405 ac- 
cording to the equation; 

ERRpts = Av-pts - A A . PTS . 

The magnitude of the error signal is examined 406 
to determine if it is greater than one half an audio 
frame interval. If it is greater than one half an audio 
frame interval, the error signal is checked for polar- 
ity 407. If the polarity is positive, the current audio 
frame is repeated 409. If it is negative the current 
audio frame is skipped 408. After skipping or re- 
peating a frame the system iterates back to the 
start position to wait for the next occurrence of 
PTS's. 

At step 406. if the magnitude of the error signal 
is less than one half an. audio frame .interval, the 
error is examined to determine if it is greater than 
zero 410. If the error is greater than zero, the error 
is checked 412 to determine if it is less than the 
previous error signal. If it is less than the previous 
error signal, this is an indication that the system is 
converging . toward synchronization, and the syn- 
chronization control parameters are not changed. 
The system returns to the start position to wait for 
the next PTS's. Conversely, if the error has in- 
creased over the previous error signal, the audio 
system processing clock is adjusted, to lessen its 
frequency 414. 

At step 410, if the error is less than zero 
(negative), it is checked 41 1 to determine whether 
it is greater than the previous error signal. If it is 
greater than the previous error signal,. this is also 
an indication that the system is converging toward 
synchronization, and the synchronization control 
parameters are not changed. Alternatively if the 
current error signal is less than the previous error 
signal the system is moving further out of synchro- 
nization and the audio processing clock frequency 
is increased 413. After processing steps 412 and 
413 the system returns to wait for the next occur- 
rence of PTS's. It will be noted in this* example, 
that the system performs only coarse adjustments 
via skipping or repeating audio frames until the A/V 
synchronization error is reduced to less than one 
half an audio frame interval. 

In an alternative embodiment, the filtered error 
signal is compared against a predetermined thresh- 
old related to the size of respective audio frames. If 
the error signal is less than the threshold, indicative 
that the audio-video timing error is less than an 
audio frame, the error signal is coupled to the 



audio time base circuit 21 5. wherein it is utilized to 
adjust the frequency of the audio signal processing 
(decompression) clock. Alternatively, if the error 
signal is greater than the threshold, the error signal 

5 may be divided by the audio frame interval to 
determine the number of audio frames that the 
audio and video signal are misaligned. The integer 
portion of the quotient is applied to the audio 
decompressor to condition the audio decompressor 

w to skip or. repeat that number of audio frames. The 
polarity of the error signal will determine whether 
audio frames should be skipped or repeated. Nomi- 
nally the decompressed data is arranged in a buff- 
er memory prior to being output, hence skipping or 

is repeating audio frames is a simple matter of con- 
trollably enabling the memory read/write com- 
mands. 

The fractional portion of the quotient is coupled 
to the audio time base circuit 215, where it is 

20 utilized to adjust the audio processing clock to fine 
tune the A/V synchronization. 

The rate of production of audio PTS's is pro- 
portional to the processing speed of the audio 
decompressor. The processing speed of the audio 

25 decompressor is directly proportional to the fre- 
quency of the clock signal used to operate the 
audio decompressor. If the clock frequency of the 
audio decompressor is independent of the clock 
used to operate the video decompressor, and is 

30 finely adjustable then the relative rate of occur- 
rence of the audio and video PTS's can be ad- 
justed and the A/V finely synchronized. 

FIGURE 5 is a first embodiment of circuitry for 
generating an adjustable audio processing clock 

35 signal. In FIGURE 5 a voltage controlled oscillator, 
503. is connected in a phase locked loop including 
elements 500 and 501. The output of the oscillator 
is coupled to one input of a phase detector 500. 
The system clock is coupled to a second input of 

ao the phase detector 500 via a binary rate multiplier 
505. A phase error signal developed by the phase 
detector is low pass filtered in filter 501. This 
smoothed phase error signal is coupled to the 
control input terminal of the oscillator to condition 

^5 the oscillator to oscillate at the same frequency and 
phase as the output of the binary rate multiplier. In 
this example the system clock frequency is ap- 
proximately 27 MHz and the desired audio pro- 
cessing clock frequency is approximately 1/380 of 

so the 27 MHz signal. A control signal from the pro- 
cess element 216 is coupled to a control input port 
of the binary rate multiplier to control its output 
frequency. This control signal is selected to repre- 
sent a. nominal division ratio of 1/380, but modu- 

55 lates about this value so as to condition the output 
of the binary rate multiplier to produce an output 
frequency commensurate with the instantaneous 
desired audio processing rate. 
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FIGURE 6 illustrates a second example of ad- 
justable audio processing clock generation. In this 
example, a constant clock frequency, from the sys- 
tem clock generator 208 or some other stable 
oscillator, is applied to one input of a binary rate 
multiplier 600. The binary rate multiplier is con- 
trolled by a' control signal from the process ele- 
ment 216 to generate a nominal desired audio 
processing clock frequency. Responsive to the A/V 
error signal the process element 216 alters this 
control signal to condition the binary rate multiplier 
to either increase or decrease the nominal audio 
processing clock frequency. 

A further alternative audio signal processing 
rate changing apparatus (not shown) may include 
an' oscillator for providing a nominal maximum 
audio processing clock frequency and a gating 
circuit. The oscillator is coupled to the audio pro- 
cessing circuit via the gating circuit. The gating 
circuit is controlled by the process element 216 to 
excise ones of the oscillator output pulses to pro- 
vide, on average, a desired audio processing clock 
frequency. 

An alternative embodiment of the synchroniz- 
ing apparatus, indicated by the dashed arrow in 
FIGURE 2. may be arranged to skip or repeat 
video frames to effect synchronization. Alternatively 
video frames may be skipped for the audio leading 
(lagging) video and audio frames may be skipped 
for the audio frames lagging (leading) video. How- 
ever, in the preferred embodiment, the audio 
frames are skipped/repeated for audio lagging and 
leading video respectively. 

Claims 

1. Apparatus for providing synchronized, repro- 
duced audio and video signals from recovered 
compressed audio and video signal, which 
audio and video signals include respective 
time stamps PTSaud and PTSvid determined 
at predetermined intervals and related to an 
encoder system clock, said apparatus char- 
acterized by: 

a source (200) of said recovered com- 
pressed audio and video signals, and a source 
(208, 37) of reference frequency; 

a counter (208, 36) responsive to said ref- 
erence frequency for providing a sequence of 
values modulo N where N is an integer; 

decompression apparatus (214) responsive 
to said recovered compressed video signal for 
providing decompressed video signal and said 
time stamp PTSvid: 

decompression apparatus (212) responsive 
to said recovered compressed audio signal for 
providing decompressed audio signal and said 
time stamp PTSaud; 



means (222, 217) for capturing values 
(LVS) of said sequence at predetermined times 
related to the provision of said time stamps 
PTSvid and for generating the difference (A v - 

s pts) of said values LVS and corresponding time 

stamps PTSvid; and (220. 218) for capturing 
values (LAS) of said sequence at predeter- 
mined times related to the provision of said 
time stamps PTSaud and generating the dif- 

w ference (A a - P ts> of said values LAS and cor- 

responding time stamps PTSaud; and 

means (219, 216. 215) responsive to said 
differences Av-pts and A A - PT s for synchroniz- 
ing the decompressed audio and video signals. 

is 

2. The apparatus set forth in claim 1 character- 
ized in that said means for synchronizing the 
decompressed audio and video signals in- 
cludes means (216, 212) for skipping/repeating 
20 predetermined intervals of decompressed 

audio signal responsive to the difference the 
between values Av-pts and A a- pts exceeding 
a predetermined value. 

25 3. The apparatus set forth in claim 1 character- 
ized in that said means for synchronizing the 
decompressed audio and video signals in- 
cludes means (216.214) for skipping/repeating 
predetermined intervals of decompressed vid- 

30 eo signal responsive to the difference the be- 

tween values Av-pts and A a- pts exceeding a 
predetermined value. 

4. The apparatus set forth in claim 1 character- 
OS ized in that said decompression apparatus 

(212) responsive to said recovered com- 
pressed audio signal provides decompressed 
audio signal at a rate determined by a pro- 
cessing (215) clock signal and said means for 
40 synchronizing the decompressed audio and 

video signals further includes means (216) re- 
sponsive to the differences (A v -PTsr(A A -PTS). 
for variably adjusting the rate of said process- 
ing clock signal. 

<s 

5. Apparatus for providing synchronized, repro- 
duced audio and video signals from recovered 
compressed audio and video signal, which 
audio and video signals include respective 

so time stamps PTSaud and PTSvid determined 

at predetermined intervals and related to an 
encoder system clock, said apparatus char- 
acterized by: 

a source (200) of said recovered com- 
55 pressed audio and video signals, and a source 

(208) of a local clock signal; 

decompression apparatus (214) responsive 
to said recovered compressed video signal for 
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providing decompressed video signal and said 
time stamp PTSvid; 

decompression apparatus (212) responsive 
to said recovered compressed audio signal for 
providing decompressed audio signal and said s 
time stamp PTSaud; 

means (217, 218, 220, 222) for determin- 
ing, in cycles of said local clock signal, the 
time T between occurrences of corresponding 
time stamps PTSaud and PTSvid; io 

means (219) for calculating the difference 
between values of corresponding time stamps 
PTSaud and PTSvid and comparing such dif- 
ference with said time T to generate an A/V 
synchronization error signal; and is 

. means (215, 216)responsive to said A/V 
synchronization error signal for synchronizing 
the decompressed audio and video signals. 

The apparatus set forth in claim 5 character- 20 
ized in that said means for determining the 
time T comprises means including a counter 
for counting cycles of said local clock signal 
and providing a value representing said time T 
which is equal to the number of cycles of said 25 
clock signal occurring between the occur- 
rences of said corresponding time stamps 
PTSaud and PTSvid. 

The apparatus set forth in claim 6 character- 30 
ized in that said means including a counter 
comprises: 

a counter responsive to said local clock 
signal for providing a sequence of count values 
modulo N where N is an integer; and ' 35 

means responsive to an occurrence of said 
time stamps PTSaud for capturing a first count 
value and responsive to the occurrence of a 
corresponding time stamp PTSvid for captur- 
ing a second count value; «o 

a subtractor for determining the difference 
between said first and second count values. 

The apparatus set forth in claim 5 character- 
ized in that said recovered compressed audio *s 
and video signal includes system clock re- 
ferences (SCR's), and said apparatus further 
includes means responsive to said system 
clock references for generating said local clock 
signal synchronous with said encoder system so 
clock. 

The apparatus set forth in claim 5 character- 
ized in that said respective decompression ap- 
paratus provide decompressed video signals in 55 
predetermined intervals and decompressed 
audio signal in further predetermined intervals, 
and said means for synchronizing includes ap- 



paratus responsive to said A/V synchronization 
error signal for skipping or repeating ones of 
said predetermined intervals of said decom- 
pressed audio signals. 

10. The apparatus set forth in claim 9 character- 
ized in that said decompression apparatus for 
providing decompressed audio signal includes 
a source of a processing clock signal for op- 
erating said decompression apparatus and said 
means for synchronizing includes further ap- 
paratus responsive to said A/V synchronization 
error signal for varying the frequency of said 
processing clock signal. 

11. The apparatus set forth in claim 10 character- 
. ized in that said apparatus for varying the 

frequency of said processing clock signal in- 
cludes a binary rate multiplier. 

12. The apparatus set forth in claim 7 character- 
ized in that said respective decompression ap- 
paratus provide decompressed video signals in 

• predetermined intervals and decompressed 
; audio signal in further predetermined intervals, 
and said means for synchronizing includes ap- 
paratus responsive to said AA/ synchronization 
error signal for skipping or repeating ones of 
said predetermined intervals of said decom- 
pressed audio signals. 

13/ The apparatus set forth in claim 12 character- 
■ ized in that said decompression apparatus for 
1 providing decompressed audio signal includes 
. a source of a processing clock signal for op- 
erating said decompression apparatus and said 
. means for synchronizing includes further ap- 
paratus responsive to said A/V synchronization 
error signal for varying the frequency of said 
processing clock signal. 

14. Apparatus for providing synchronized, repro- 
duced audio and video signals from recovered 
compressed audio and video signal, which 
audio and video signals include respective 
time stamps PTSaud and PTSvid determined 
at predetermined intervals and related to an 
encoder system clock, said apparatus compris- 
ing respective decompression apparatus for 
decompressing said compressed audio and 
video signals and producing said time stamps 
PTSaud and PTSvid; and means responsive to 
the occurrence of said time stamps PTSaud 
and PTSvid for measuring the time between an 
occurrence of a PTSaud and the next occur- 
ring PTSvid and adjusting the decompression 
rate of at least one of said decompression 
apparatus such that the time between an oc- 
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currence of a PTSaud and the next occurring 
PTSvid substantially equals the difference in 
values of said PTSaud and the next occurring 
PTSvid. r 
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